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ABSTRACT 

A longitudinal study examined the factors affecting 
the long-term success of a Midwestern tech prep consortium consisting 
of one community college and four counties with a mix of rural, 
urban, and suburban communities. The study was designed to identify 
the effects of systematic change over time from the following 
perspectives: students' skill gains; how and what teachers teach; the 
broader school environment in which instruction occurs; 
district-level, parental, and business support for tech prep; and the 
impact of state and federal policies ^on local-level tech prep 
programming. The following data collection methods were used: review 
of historical documents; analysis of educational records; participant 
observation; field observation of business classrooms in a high 
school, comprehensive high school, vocational center, and community 
college; surveys; and interviews. Six months after the study had 
begun, a number of aspects of implementation of systematic change 
were identified. The consortium seemed to be nearing the "takeoff" 
point in the process of communicating innovation through specified 
channels over time to members of the community's social system. Early 
adopters of the tech prep system had discovered how to communicate 
information about the program and had identified and enlisted the 
support of key leaders. (Contains 20 references.) (hN) 
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Evaluating Systemic Change in School-to-Work Initiatives 



Phyllis T. H. Grummon 
Michigan State University 

This study considers the range of factors that may influence the long term success of 
systemic reform related to education for employment. It presents the initial findings of a 
multi-year project to investigate the success of a specific school-to-work initiative (Tech 
Prep)in a community. It considers this change in light of the literature on the diffusion of 
innovations and the process of large scale system change. 
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EVALUATING SYSTEMIC CHANGE IN SCHOOL-TO-WORK INITIATIVES 

Sometime in the last twenty years or so, the rules about how young adults move from school 
into employment changed. Prior to 1970, high school graduates and even those without a diploma, 
could expect to find employment relatively easily and to earn enough money to maintain a middle class 
existence. While this was most true immediately following World War II, the ability of youth to find 
well-paying work changed little until the late 1960s and early 1970s. What has happened since then, is 
that real earnings for high school graduates have fallen by an average of 28% (Education Writers 
Association, 1990; Policy Information Center, 1990). A pattern has emerged of youth between the ages 
of 16 and 25 finding employment primarily in low paying ($4.25 - $6.00 per hour) jobs in what is calted 
the secondary labor market (Hamilton, 1990). While a young person may be employed full-time 
following high school in such jobs, it is virtually impossible to save money, start a family, or even move 
away from home. The shift from manufacturing to service sector jobs has meant that even those with a 
high school diploma are unlikely to find well paying work, while those without a diploma are very 
likely to be unemployed. 

In addition to the loss of manufacturing jobs, those jobs that do pay middle class wages have 
changed too. By-and-large, far more skills, both academic and interpersonal, are needed to enter and 
advance in jobs. The ability to work with a variety of technologies, to use statistics and mathematics 
to analyze problems, to read and interpret complex manuals, and to work in teams are now routinely 
required in all sectors of the economy. Any number of surveys of employers, both formal and informal, 
reveal they are seeking employees with an ability to learn even more than those with any specific skills 
(Berryman, 1993; Carnevale, Gainer, & Meltzer, 1990; Mehrens, 1989). So, students leaving high school 
are likely to find that what they learned will be insufficient to allow them to enter more than a 
minimum wage job, essentially the same job they held in high school. Given that the majority of 
students either do not go on to postsecondary education or do not complete that education, the need 
for the educational system to respond is critical. 

The problem of how to help youth successfully gain entry into the workforce is multifaceted. 
The policy emphasis has bejn on the development of programs to increase the skills of students leaving 
high school so they can obtain more "high skill, high wage" jobs (Grummon, In Press). Many of the 
public sector initiatives in this arena were created following research and reports from public study 
commissions (e.g.; W. T. Grant Foundation, 1988; Commission on the Skills of the American Workforce, 
1990; Secretary's Commission on Achieving Necessary Skills, 1991). This research has been largely 
descriptive in nature. The programs developed from these reports have relied on programs developers' 
general sense of what might help students make the transition to work. 

One specific initiative at the federal level was included in the 1990 Re-authorization of the Carl 
Perkins Act. The Tech Prep Education Act stated: 

"...in recognition of the recent changes in the labor market, and the challenge of worldwide 
economic competition, there is a significant need to prepare youths for success in the ever 
changing technological workplace. This preparation can be provided through a 4-year 
educational program grounded in,the development of comprehensive instruction based on 
articulation agreements between secondary and postsecondary institutions." (pp. 38-39) 

In fact, the Perkins Vocational and Applied Technology Education Act provided funding for a number 
of changes that were already taking place in school districts and community colleges around the 
country. Articulation agreements, such as 2 + 2 programs, had already begun to be negotiated between 
secondary and postsecondary institutions in many communities. What the Perkins Act brought to the 
table, however, were some clear expectations that articulation agreements would not be enough to 
create the type and quality of changes sought by the legislation. In order to truly meet the spirit of the 
law, and the changes desired by businesses and individuals, secondary and postsecondary institutions 
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would have to undergo systemic change. This change focuses ^imarily on the move from activity- 
based success in education (i.e.; the accumulation of Carnegie Units and student credit hours) to 
competency or skill-based outcomes. For Tech Prep to work as designed, postsecondary and 
secondary institutions must not only change how they instruct and assess students, they must also 
make decisions jointly that reflect those changes. 

As presented in federal guidelines and interpreted by the state, Tech Prep has a number of 
required components (Dornsife, 1992). In Michigan, the site of this study, the state has emphasized the 
following components in its application guidelines to consortia; work-based learning opportunities, 
sequences of outcomes/skills for each occupational area, integration of academic and occupational 
instruction, professional development, Tech Prep integrated into other reform efforts, learner support, 
and educational development plans and portfolios for all students (Michigan State Board of 
Education, 1993). 

Tech Prep, as a specific school-to-work initiative, takes place within the broader context of 
school reform. A district or school that ;s involved in Tech Prep is also likely to be working on some 
other form of systemic reform. The number and diversity of programs designed to help schools 
improve is readily apparent from reading either the popular or research literature. While each of these 
programs has a different slant on school reform, they tend to emphasize the need for students to 
engage in more contextualized, activity-based learning (e.g.; Berryman, 1993; Copa & Pease, 1992; 
Peterson & Knapp, 1993; Prestine & Bowen, 1993; Sizer, 1985, 1992). Thus, many of the practices 
advocated by school reform or school improvement are similar to the ones discussed in school-to-work 
initiatives. The state has recognized this and encouraged consortia to integrate the various initiatives 
as much as possible. 

The Study 

This paper presents the results of the first six months of a longitudinal study of one Tech Prep 
consortia in a Midwestern community. The consortium includes four counties with a mix of rural, 
urban, and suburban communities. The consortium includes a single community college. One county 
has been leading the efforts for integrating systemic change with Tech Prep and preparation for the 
workplace. That county has been the focus of initial data collection and serves as the pilct for 
methods to be used throughout the consortium in subsequent years. 

In order to capture the effects over time of systemic change, this study looks at change from a 
number of perspectives (see Figures 1 and 2). . irst, students and teachers are the ultimate focus of 
most change efforts. As discussed above, one or the primary driving forces for programs is to help 
individual students gain skills for use in the workplace. How and what teachers teach, then, becomes 
the vehicle for imparting those skills. What curriculum they use, what instructional techniques, their 
background and commitment to preparation for the workplace, all become factors in determining if a 
school-to-work program will succeed. However, classroom instruction happens within the context of a 
broader school environment. Such factors as the support of the principal, the reaction of other teachers 
to changed instructional techniques, the resources available to the teacher (e.g.; equipment, counselors, 
professional development, etc.) and curriculum all affect what can and does happen in a classroom. 

Outside of the building, district level support, parental support, and the make up of the Tech 
Prep consortium can all affect the ability of change to be initiated and sustained. For secondary school 
students, the support of the community college and the business community are also important. In 
particular, for Tech Prep, the overall philosophy of the community college and the level of commitment 
from faculty and top administrators affect how articulation between secondary and postsecondary 
learning occurs. The business community plays a role for both community college and secondary 
students. Local business involvement may come in a number of ways, including collaboration on 
curriculum development, guaranteed jobs, internships, apprenticeships, or other skill development 
activities. At a minimum, local businesses must confirm for students that the skills being taught in 
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classrooms are in fact ones they will use for selection. Finally, state and federal policies affect how 
Tech Prep is enacted at a local level Funding for local initiatives comes from federal sources. State 
agencies are charged with distributing funds, establishing monitoring practices, coordination of efforts, 
and generally interpreting federal guidelines to enable consortia to meet federal requirements for 
funding. 




Figure 1: Evaluation Factors in Secondary Tech Prep 
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Figure 2: Evaluation Factors in Postsecondary Tech Prep 

In order to understand how systemic change is enacted, information was and will be collected 
at each of these levels. Data collection methods include review of historical documents, analysis of 
educational records, participant observation, field observation, surveys, and interviews. The methods 
used vary depending upon the level of the system under analysis and the type of information available. 
It is expected that over time a rich and complete record will be constructed of the ways in which Tech 
Prep has evolved in this consortium. While those interested in reforrr 1 have often looked at one piece of 
the system, rarely have studies attempted to gather data from each of those pieces (Wang, Haertel, & 
Walberg, 1993). It is hoped that by paying attention t^ the many factors that impinge on the 
effectiveness of a reform effort, the study will be able to shed light on how those factors interact and 
which factors may be more powerful. Given that, it is critical to note that time plays a key role. 
Factors that may be important at the start of the reform process, may not be as significant after a 
period of time. The diffusion of innovation literature would support time as a vital dimension in any 
change process (Rogers, 1983). 
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Classrooms 



Four different classrooms are being observed as part of this study. The classrooms represent 
the range of settings in the consortium where Tech Prep is being enacted for students and teachers. 
They are an applied mathematics class in a high school, a business class in a comprehensive high school 
(i.e. a high school that offers vocational instruction on its premises), a business class at the vocational 
center, and two business classes at the community college. The classrooms are all part of the business 
occupational cluster in the Tech Prep consortium. Each of the teachers has been involved in the 
consortiums business curriculum restructuring process for the past year. They are considered 
exemplary teachers by the consortium's Tech Prep coordinator and were recruited on that basis. Part 
of that view of the teachers is based on their involvement in the business curriculum team. They have 
also received additional professional development, and in some instances, visited a school in Colorado 
that is considered a model for what the consortium is attempting. 

Each high school classroom will be observed for one week. The business classes at the 
community college will be observed over the course of two weeks. Two observers will take field notes 
and tapes of the activities in the classrooms. They will be using the typology developed by bt sz, 
Ramsey, Eden, DaVanzo, Farris, and Lewis (1993) for coding field notes. This method was developed 
in observation* cf vocational classrooms trying to teach generic skills and has direct relevance to the 
goals of Tech Prep. Some of the codes will be modified for the community college classrooms (e.g.; 
S postsecondary focus). Notes will also be made related to the classroom environment. These will 
include attention to the factors listed below. 

°What is the physical design of the classroom? How flexible is it? 
°What is the environment of the class (social interactions, task behaviors, etc.)? 
°What resources are available to the class (books, materials, people, equipment, etc.)? 
°How are resources used by students? teachers? others? 

In addition to observations, students in each classroom will be asked to complete a brief 
questionnaire. The questionnaire includes items on their plans after graduation, preferred learning 
styles, how they use skills learned in the class, and their present employment. Teachers have each had 
a brief preliminary interview and will be interviewed more extensively following classroom 
observations. Interview questions will include: 

°How does the teacher approach teaching? (philosophy of teaching, pedagogy) 

°What are the teacher's instructional tactics and strategies? 

°What are the teachers goals, as perceived by the teacher? 

°On what knowledge, skills, and attitudes does the teacher focus instruction? 

°What are the teacher's learning expectations for students? 

°What strategies does the teacher use to help students think about non-school applications for 
what they are learning? 

°What specific instructional work does the teacher expect students to accomplish? 
°How does the teacher evaluate the students? 
°What is the teacher's experience (work history)? 

Information from the interview will be considered in light of the classroom observations to gain a sense 
of how the teacher's actual classroom practices reinforce or detract from their stated goals. 
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Supporting Structures 



Each of these classrooms is located in a different building and district, or in the case of the 
community college, a department under a dean. The support that teachers receive in the classroom and 
the general support for Tech Prep in the school or college may affect the likelihood that the teachers will 
continue to be involved in innovative instructional practices (Prestine & Bowen, 1993). Interviews and 
participant observations will be the primary methods of data collection during the pilot phase. 
Interviews will be conducted with principals, counselors, and teachers identified as supportive of Tech 
Prep, neutral, or unsupportive. The goal of these interviews will be to gauge how Tech Prep is 
perceived in the specific settings, as well as the overall level of resources available in the school/ college 
to support Tech Prep activities (e.g.; counseling, computers, labs, professional development time, etc.). 

The role of the Tech Prep coordinator is also key to the initiation and maintenance of Tech Prep 
as an innovation. In the case of this consortium, there are really two people who function in the role, 
one formally and the other informally. The official coordinator works at the community college. The 
'informal' coordinator works at the intermediate school district. Both individuals have been 
interviewed and will continue to be key informants throughout the research. 

The environment outside the building or department also may affect how Tech Prep is enacted 
and its ability to sustain itself as an innovation (Rogers, 1983; Prestine & Bowen, 1993). The-support 
of superintendents, deans and presidents, parents, and the larger Tech Prep consortium may all play a 
role. The county in which the school districts are located has formed a special advisory group to help 
oversee the integration of such efforts as Tech Prep across the county. This group consists of 
representatives of each district in the county and includes superintendents, assistant superintendents, 
and curriculum specialists. Each member of this group will be interviewed and records of their 
meetings since the inception of the group two years ago are being reviewed for key events and 
decisions. Interview questions will be similar to the ones conducted with principals. The dean of the 
community college responsible for the business class being observed will be interviewed, as will be the 
president. 

In the case of Tech Prep, support of the local business community and parents is vital for its 
success. Parents must be willing to view a future for their children that does not necessarily include a 
four year degree. Like students, they must be able to accept that academic performance in high school 
has a direct bearing on students' future job prospects. Marketing Tech Prep, to parents, students, and 
businesses is considered a key component by the state. As described above, businesses also can play a 
significant role in the ultimate success of a Tech Prep program. In this consortium, local business 
representatives worked with teachers in restructuring the business curriculum. Business and parent 
representatives will be interviewed during the study to ascertain their perceptions of Tech Prep and 
education for employment in general. 

The final supporting structure to be considered is the role of state and federal policies related to 
Tech Prep and school-to-work. While Wang, et al. (1993) have indicated that these tend to play a 
peripheral role in student achievement, they are likely to be more central in the enactment of Tech Prep. 
The federal government established the concept of Tech Prep in its re-authorization of funds under the 
Carl Perkins Act. The recent passage of the School-to- Work Opportunities Act has created what may 
be either a rival or complementary initiative. It is unclear at this point how the consortium under study 
will respond to this new program. The two programs, however, are administered out of different state 
agencies, so it is likely that at least some conflict will occur. How such conflicts are resolved may 
affect the degrees of freedom that local consortia have in developing Tech Prep and other school-to- 
work initiatives. 
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Results 



A number of aspects of the implementation of a systemic change have been revealed through 
initial data collection efforts in one county. The importance of a visionary leader has been particularly 
apparent. Through interviews and review of historical documents, it is becoming clear that without the 
involvement of the 'informal' Tech Prep coordinator, the implementation of Tech Prep might look very 
different. This leader has considerable longevity in the community and the intermediate school district. 
He helped to start the vocational center and acted as its principal until two years ago. At that time, he 
took a position as an assistant superintendent in the intermediate school district with the goal of 
promoting a broader acceptance of education for employment. He will retire at the end of June, but 
plans to continue working one-third time on this initiative. How his retirement will affect the 
implementation of the Tech Prep remains to be seen. However, he has worked to promote the 
involvement of as many people as possible along the way so that he is not see as the sole driver of the 
change. 

This leader has identified a number of strategies for encouraging systemic reform, which he sees 
as going beyond the need for education for employment. M. has taken over 85 school personnel from 
the county to a site in Colorado which serves as a model for how they see their schools being 
restructured. This model includes hands-on technology laboratories, active learning strategies, and 
integrated academic instruction. This has enabled a significant number of teachers to literally see the 
vision and to own it for themselves. M. has worked with a variety of committees to ensure that 
professional development opportunities support this vision of a restructured school. He has brought in 
national speakers and sponsored consultants to work on such issues as assessment. He identifies 
readings and broadly circulates them. M. also realized that just bringing in speakers and sponsoring 
workshops could not sustain change at the building level. So, he began facilitating teams of teachers, 
such as those involved in restructuring the business curriculum, on a regular basis. M. is the chair of the 
Superintendents Advisory Council (SAC), the group that acts to integrate initiatives across the county, 
including special education, drug-free schools, infants and pre-school programs, technology education, 
and Tech Prep. 

The SAC described the history of education for employment in a focus group session that 
highlights the breadth with which the county is approaching the initiative. As they began the 
discussion of this history, the group identified a single event as the trigger for interest in education for 
employment, a presentation by a business owner to the county's school board association in 1990. 
They perceived that the presentation triggered discussions among superintendents about the problem of 
students leaving school without a job and no plan for further education. It soon became apparent to 
the group, however, that while the presentation may have been the catalyst, prior events and activities 
had laid the groundwork for an interest in education for employment. Some of those events/activities 
included a manufacturing business task force, a Business-Education partnership, and work in 
individual buildings and classrooms to bring business and education closer together. One assistant 
superintendent identified a state initiative from the early 1970s that focused on career education and 
life role competencies as being a precursor for thinking about how schools help prepare students for the 
workplace. Important to moving the ideas in the community and schools into reality was the funding 
of positions at the intermediate school district starting in 19S9. These positions include ones for 
coordinating efforts to restructure academic subjects (mathematics initially), work with Business- 
Education Partnerships, and technology education and career guidance. 

One of the primary foci for change has been the introduction of technology education and the 
concomitant use of technology in applied learning across the curriculum. In the fall of 1990, the 
superintendents of the ten school districts in the county agreed to introduce technology education 
(similar to that in the Colorado model) and computer labs to the middle schools. The intermediate 
school district provided funding for the purchase of technology labs. Three relatively small districts 
started the process in 1991-92. In 1993-94, six more districts committed funds and space to the 
building of labs, including the largest district in the county. The last district, the wealthiest, will build 
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their lab in 1994-95. The SAC perceives that the changes in learning at the middle school (i.e.; 
introduction of learning labs, integrated technology education, professional development of teachers) as 
being completed by the end of the 1995-96 school year. Tiieir hope is that the changes in the middle 
school will act to push the high schools to change. The believe that middle school students, who have 
been taught more interactively, and their parents will force the high schools to change. 

The SAC has more concerns about changing instructional practices in the high school than in the 
middle schools. They anticipate those changes taking four to five years. Issues of certification, staff 
development, the phasing out of present courses, and how to structure access to computer labs (e.g.; 
centrally, by content areas) all must be confronted. Unlike the middle schools that had a fairly uniform 
curriculum and introduction process, "hey see each high school as needing a unique plan. Funds for 
computers and redesign of schools have been committed by local districts and the intermediate school 
district, so it is likely that the changes will actually proceed. 

The Superintendents Advisory Council identified factors for success that were similar to those 
used by the visionary leader. They stressed the importance of early staff development, as well as 
committing sufficient time to planning and preparing for change. Other factors they indicated were; 
important for the success of systemic reform were a clear vision, key people (e.g.; administrators, 
superintendents, etc.) were committed visibly to the changes, that people (change agents) spend 
sustained time with teachers working in teams after staff development workshops, and that there be 
enough people who believe in the change so that the loss of key people does not create too many 
problems. 

Initial discussions with faculty and staff at the community college indicate that there is much 
less awareness of the specifics of Tech Prep and the types of changes being sought by K-12 districts. In 
part, the community college has always been committed to education for employment through its 
occupational education program. Thus, there is a belief that they are already doing what they need to. 
Where there is ongoing discussion about change is in how to recognize the increased skills of students 
who have completed Tech Prep in secondary school as they enter specific community college courses 
and programs. Discussions are occurring about whether to offer credit or create higher level courses for 
such students. Since Tech Prep graduates are likely to always represent only part of the college's 
enrollment, the college will not be changing as much as the secondary schools. At this point, however, 
the involvement of the community college's faculty in professional development or awareness of Tech 
Prep has been limited primarily to the Department of Office Administration, the Tech Prep 
Coordinator, and the Dean of Applied Arts and Sciences. The Tech Prep Coordinator has been 
working successfully with at-risk populations (chronically unemployed) using Tech Prep models. 

Informal observations in the classrooms have been limited. Two of the high school classrooms 
use computer software to teach students business practices. One classroom also has an ancillary 
teacher to supplement instruction on mathematics. Instruction is focused on skill acquisition in these 
classrooms and the teachers individualize programs through a sequence of modules that must be 
mastered. One teacher described the students as largely unmotivated until they become seniors and 
realize they need to have skills after graduation. The other teacher did not identify that as an issue. 
The applied mathematics class uses the CORD (Center for Occupational Research and Development) 
material. This classroom is of interest because most of the students have been together for two years 
with the same teacher. The community college classrooms have not been observed yet. One takes place 
during the day and one at night which gives them different populations of students. 
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Discussion 



The results of initial data collection seem to indicate that this county is nearing or at the point in 
the diffusion of innovation process called "take off (Rogers, 1983). Rogers (1983, p 10, emphasis in 
original) defines diffusion as "the process by which (1) an innovation (2) is communicated through certain 
channels (3) over time (4) among the members of a social system. 1 ' Early adopters, including the visionary 
leader, have found ways to spread information about the innovations (e.g.; changed instructional 
strategies, use of technology in learning, need for technology education, applied learning strategies, etc.) 
throughout the districts of the county over the past two years. A variety of techniques have been used. 
The three principal ones are showing people a model (trips to Colorado), staff development, and 
sustained facilitation of teams of teachers /school personnel working on the change. 

In addition to communicating more generally about both the need for change and the types of 
change sought, the change agents have also identified and enlisted the support of key leaders. 
Discussions of systemic reform identify the importance of not just a critical mass of teachers, but also 
the support of leaders for the success of a change (Prestine & Bowen, 1993). The problems that can 
arise when such leaders leave a district were also identified. Fortunately for most districts, 
administrators tended to be stable and/or to be promoted from within the district from people who 
were already committed to the change. The personnel interviewed, however, all indicated the need to 
continue to broaden the base of support for the innovation, particularly among community and 
business leaders. 

Future research activities will focus on a number of issues, first will be the extent to which 
innovations are actually enacted in classrooms. While teachers may receive staff development and be 
committed to changing their instructional practices, there are a number of environmental factors that 
may inhibit that happening. Students are not generally accustomed to more active/ construe tivist 
learning strategies and so create a "pull" for more familiar practices. The physical layout of most 
classrooms, as well as scheduling of classes, make it more difficult to engage in applied learning. There 
are no schools where all of the teachers are committed to, or familiar with, the changes being sought. 
This creates tension among those who are trying new practices and those who prefer more familiar J 
techniques. Depending upon the influence level and number of adopters, non-adopters may present a 
significant barrier to adoption of the innovation. Thus, it will be important to continue to observe the 
teachers who have initially volunteered for the study as additional teachers are enlisted in different 
subject areas. 

As important as the extent to which innovations actually appear in classroom practices will be 
the rate at which changes are adopted across schools within the county and across the four counties 
and community college within the consortium. The rate at which the innovation will diffuse is likely to 
depend not only on the number of adopters, but also such factors as support from state and federal 
policies, the business community, parents, and even the national and international economy. While we 
like to think of change as orderly and planned, in fact it can often be affected by serendipitous events 
outside of the control of change agents or planners. Thus, this research will also attempt to monitor 
and measure the effects of changes in the outer shells of Figures 1 and 2 as they affect the planned 
changes in the consortium. 

A final note, this research is only in its preliminary stages. Just as those involved in the 
innovation understand that it will take many years to realize their vision, so too, the results of those 
efforts and the processes by which they were achieved will not be revealed until much more data is 
collected. By working with the those involved in the change, we hope that our research can help them 
make informed decisions about their activities, as well as contributing to the general knowledge about 
how large scale change becomes enacted at all levels of the system. 
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